TPAMI 112351 Interactive optimization of 3-D shape and 2-D correspondence using multiple geometric constraints via POCS

The traditional approach of handling motion tracking and structure from motion (SFM) independently in successive steps exhibits inherent limitations in terms of achievable precision and incorporation of prior geometric constraints about the scene. This paper proposes a projections onto convex sets (POCS) framework for iterative re nement of the measurement matrix in the well-known factorization method to incorporate multiple geometric constraints about the scene, thereby improving the accuracy of both 2-D feature point tracking and 3-D structure estimates. Regularities in the scene, such as points on line and plane and parallel lines and planes, that can be interactively identi ed and marked at each POCS iteration, enforce rank and parallelism constraints on appropriately de ned local measurement matrices, one for each constraint. The POCS framework allows integration of the information in each of these local measurement matrices into a single measurement matrix that is \closest" to the initial observed measurement matrix in Frobenius norm, which is then factored in the usual manner. Experimental results demonstrate that the proposed interactive POCS framework consistently improves both 2-D correspondences and 3-D shape/motion estimates, and similar results can not be achieved by enforcing these constraints as either postor pre-processing. keywords: Geometric constrained shape recovery, structure from motion, interactive optimization, the factorization approach, projections onto convex sets (POCS).